Learning Image Structures for Optimizing Disparity Estimation

We present a method for optimizing the stereo matching process when it is applied to a series of images with similar depth structures. We observe that there are similar regions with homogeneous colors in many images and propose to use image characteristics to recognize them. We use patterns in the data dependent triangulations of images to learn characteristics of the scene. As our learning method is based on triangulations rather than segments, the method can be used for diverse types of scenes. A hypotheses of interpolation is generated for each type of structure and tested against the ground truth to retain only those which are valid. The information learned is used in finding the solution to the Markov random field associated with a new scene. We modify the graph cuts algorithm to include steps which impose learned disparity patterns on current scene. We show that our method reduces errors in the disparities and also decreases the number of pixels which have to be subjected to a complete cycle of graph cuts. We train and evaluate our algorithm on the Middlebury stereo dataset and quantitatively show that it produces better disparity than unmodified graph cuts.